I wouldn't say it's idiotic. Buying any used car is a gamble. It's probably a good warning sign of prior ownership standards though. One skipped service is unlikely to be in too late territory, but personally I would swerve one that's skipped such an important service as the doubt can eat away at you.
So long as the shifts are smooth and there's no nasty noises pulling away, coming to a stop etc, generally that's a good sign.